hp-Variational Physics Informed Neural Networks (hp-VPINNs) [2] are a class of neural network based solvers for partial differential equations (PDEs) that use the variational form to compute residuals. They have been shown to be more accurate than vanilla PINNs, particularly for high-frequency problems when h- and/or p-refinement is used. Moreover, by eliminating the need for higher-order derivatives in residual calculation, hp-VPINNs can be more efficient than other variations of PINNs. The FastVPINNs framework [3] was designed to address several drawbacks of existing hp-VPINNs codes. The FastVPINNs code pre-computes and stores the gradient of the test functions in a 3-dimensional tensor, achieving up to a 100x speed-up over existing frameworks while solving PDEs on complex geometries. However, this poses a challenge for real-time training for solutions of dynamical systems, especially on resource-constrained computing platforms such as edge devices. In the present work, we extend the FastVPINNs framework to be compatible for training on low-memory edge devices. We achieve this by applying tensor decomposition techniques like the singular value decomposition (SVD), Tucker decompostion, CP decompostion and tensor-train decomposition (TTD). These decompositions are applied to both the trainable parameters of the network and the test function gradient tensor to obtain a low-rank approximation, thereby reducing the memory and computational requirements of the solver. For each decomposition method, we analyze its effect on accuracy and training time as a function of the percentage reduction in number of parameters. For scalar PDEs, we demonstrate that using Tucker and TTD decompositions in FastVPINNs achieves more than 95% relative accuracy compared to the baseline model, while obtaining up to 90% reduction in tensor size. Additionally, we report the time taken to train FastVPINNs on edge devices with various decomposition methods, highlighting the feasibility of their deployment for real-time applications in resource-constrained environments. For dynamical systems, this tensor decomposition-based training of FastVPINNs enables efficient deployment, making the model ideal for real-time monitoring and training in resource-limited environments.